The development of reading tests for use in a regularly spelled language

Data are presented on the development of tests of reading skill for primary school pupils in rural Tanzania. Instruction in these schools is in Kiswahili, a regularly spelled language. Using a translation of a standard reading test, children can read aloud all words once they have learned the sound– letter correspondences, regardless of comprehension. In addition, children can pass traditional comprehension tasks by decoding only some of the words. Three graded tests were developed to test children who had only some letter knowledge, could read single words, or were proficient readers. The tests required children both to decode and to understand the reading material in order to achieve high scores. The tests correlated well with scores on other educational achievement tests and showed age and school grade differences. It is suggested that these tests are useful measures of reading development in a regularly spelled language. Their adaptation to English and validation against standardized instruments are planned. A good reading test should test whether pupils can both decode and understand the words they are reading. It is difficult to construct such a test in a regularly spelled language. Readers can decode words that they do not understand, and some forms of test, such as an oral single-word reading task, allow them to read  2000 Cambridge University Press 0142-7164/00 $9.50 Applied Psycholinguistics 21:4 526 Alcock et al.: Kiswahili reading tests aloud correctly words that they do not know. The reverse is also true in any language: given a passage to read with comprehension questions to answer, children give the impression of being able to decode words that in fact they cannot read because they are able to “fill in” words that they cannot decode. In this study we developed a graded test of reading for a regularly spelled language. There is no standardized test in existence for the language we are studying (Kiswahili); the reading tests that do exist are mainly those constructed by individual teachers for use in the classroom. In the absence of standardized tests, any attempt to validate a new test must rely on good score distributions and systematic variation in test scores with years of schooling and age. To examine the alternatives that have previously been used, we review existing studies of reading in other regularly spelled languages; it seems that there are few standardized tests in such languages. This may be due to the general underserving of non-English speaking populations by standardized test development. However, it may also stem from a slightly more systematic cause: reading assessment methods used in English and other irregularly spelled languages are, by definition, unsuitable for use in regularly spelled languages. Thus, general efforts to develop tests for irregularly spelled languages will not assist those working in regularly spelled languages. Experimental studies of reading in regularly spelled languages have, in fact, generally found this to be true: the types of tests intended for use in English are not suitable for use in such languages. Many first-pass reading tests in English are oral single-word reading tests where the primary measure is the error score. Some researchers have indeed used such tests to look at error scores in regularly spelled languages. Cossu, Shankweiler, Liberman, Katz, and Tola (1988), in a study with Italian children, examined oral reading of two-syllable words from children’s first and second grade reading books and divided children into good, poor, and average readers. However, they did not report children’s scores on this test. In another experiment, these authors looked at error scores for children in the first grade, but found that children in other grades had scores that were too high to use except for error analysis (Cossu, Shankweiler, Liberman, & Gugliotta, 1995). In a study in Spanish, which is slightly less regularly spelled than Italian, Carrillo (1994) found that many children scored at ceiling, even in first grade. However, these studies did not consider the possibility that children were reading aloud words that they did not comprehend. German, which is spelled more regularly than English but not completely so, seems to be the only language to have standardized single-word reading tests (Wimmer, 1996). These tests measure not only the number of errors, but also the speed of reading (Wimmer & Hummer, 1990). This approach of looking at both speed of reading and error rates seems to be more successful in such languages. Wimmer and colleagues (Wimmer, Landerl, & Schneider, 1994) used a task that involved timed reading of both words and sentences. Porpodas (1986, 1991) timed the reading of texts in Greek, and Wimmer’s (1996) German task was similar. In the latter study, even dyslexic children made few errors; however, in another study, such children were found Applied Psycholinguistics 21:4 527 Alcock et al.: Kiswahili reading tests to be impaired on speed of reading (Wimmer, 1993). However, timed tests can be more difficult to administer than untimed tests. Other studies have used reading tests that include nonwords to examine children’s decoding accuracy. Working in Spanish, Valle-Arroyo (1989) found that there was an effect on reading accuracy of the length of a word but not, for real words, of frequency. It was concluded that children were using only the phonological route for reading. Working in Italian, Cossu et al. (1995) found that children reading words and nonwords did not confuse visually similar words and letters, but made phonologically based errors. Klicpera (1989) found that German children did make more errors on unfamiliar words and nonwords, suggesting that they were using a form of the lexical route and that there may be some frequency, or at least familiarity, effect in a regularly spelled language. In contrast to these single-word tasks, studies using text reading have tested reading comprehension. For South American Spanish-speaking children, de Manrique and Signorini (1994) used a standardized comprehension task, in addition to spelling of regular and irregular words and reading of single words used in the spelling test. A less skilled group of readers was found to be much more successful on spelling than on reading. Naslund and Schneider (1991, 1996) gave German second graders a multiplechoice text comprehension task, as well as a test where they filled in the blank in a sentence with the correct word. The single-word tasks used by this group included matching real or distractor words to a picture and choosing the correct antonym or synonym from a set of choices. But as mentioned before, text comprehension tests are inherently problematic, owing to the ability of children to fill in words they cannot read using general knowledge and expectations. In other studies, more general assessments of reading were used as part of the experimental study or as a means to separate readers from nonreaders or poor readers from good readers. Porpodas (1986) acknowledged the lack of standardized reading tests in Greek and used the class teacher to assess informally whether children were reading or not. Zucchermaglio and colleagues (Zucchermaglio, Pontecorvo, Tonucci, & Blachowicz, 1986) used a rather subjective 4-point scale for writing and reading in Italian, ranging from reading fluently without errors to refusing to read, and judged comprehension of a written sentence by whether children followed the instructions it gave. Again, such assessments seem to have many problems, not least of which is subjectivity. Hence, in developing a reading test for a variety of skill levels in a regularly spelled language that might be adaptable for use in other languages, we had several options: oral single-word reading tests measuring speed, errors, or both; continuous text reading, either timed or with some kind of comprehension measure; or a more complex test involving multiple-choice responses, nonwords, or misspelled words. None of the tests that have previously been constructed have looked at letter knowledge except indirectly as part of a word or text reading task. There is, however, the possibility of examining this aspect of children’s reading developApplied Psycholinguistics 21:4 528 Alcock et al.: Kiswahili reading tests ment and of gaining a picture of very early or prereading literacy skills. Piloting revealed that, at this early stage, many children were unable to read words, but had some letter knowledge. These factors led to our decision to devise three graded reading tests at different levels to measure basic knowledge of letters, single-word reading, and reading of extended text. Background to Kiswahili The current study took place in an area of coastal Tanzania where Kiswahili is the most common language. Kiswahili is spoken as a native language by inhabitants of the coastal areas of Tanzania and Kenya. It is the national language of Tanzania as well as the language of instruction in primary schools, although most children in rural areas away from the coastal strip speak one of a large number of vernacular languages at home. A factor taken into consideration when choosing the study area was that a single language should be the children’s first or primary language as well as the medium of instruction in primary schools. Until the early part of this century, Kiswahili was written in Arabic script; it was then officially and fairly systematically transcribed into Roman lettering (Inter-territorial Language [Swahili] Committee to the East African Dependencies, 1956). As with many other African languages, most of which have no prior written form, its orthography is constructed (not evolved) and hence is regular. Kiswahili has only five distinct vowel sounds, and thus the five vowels of written English are sufficient to transcribe all the vowels of the spoken language with a one-to-one correspondence. Consonants are the same as in English except that there is no q or x; although there is a slight phonetic variation for some consonants, each has a direct one-to-one correspondence with one phoneme. In addition, some digraphs are used (ch (/t1/), sh (/1/), th (/θ/), dh (/U/), gh (/~/), and kh (/X/)), but again these all have one-to-one correspondences. Stress is predictable, falling on the penultimate syllable without exception. Morphological features are transcribed phonologically. Hence, each written representation of a word can have only one phonological representation. Where one grammatical morpheme has more than one phonetic realization, the spelling follows the phonetic pattern, unlike in English. For example, the prefix /ma/ indicates a plural in some circumstances. This can have more than one phonetic realization, depending on the following phoneme: in makubwa ‘big things’, the prefix precedes a consonant, the phonetic realization is /ma/, and it is spelled ma; whereas in mengi ‘many things’, the prefix precedes a vowel (in this case /i/), the phonetic realization is /mε/, and it is spelled me. Contrast this with the uniform spelling ed for /id/, /d/, and /t/, which is the same grammatical morpheme (the regular past tense) in English. Due to dialect differences, the pronunciation of the phonological representations may differ in different areas of the country. In some areas voiced stop consonants are implosive and unvoiced stop consonants are explosive, but in other areas all stop consonants are explosive. Children in the study area tend to use /r/ and /l/ interchangeably. Such dialectal variations would not affect children’s ability to decode the written word because they would merely learn to associate two graphemes with one, potentially variable, phoneme. Teachers and Applied Psycholinguistics 21:4 529 Alcock et al.: Kiswahili reading tests testers realize these issues and do not penalize oral reading based on dialectal pronunciation; the child’s phonological representation would still correspond to a word in their lexicon. To illustrate, in the case of l/r substitution there is only one minimal pair in the language: kalamu ‘writing instrument’ versus karamu ‘feast’. If a child read aloud either of these words with /r/ or /l/, a tester would score the reading as correct. The child would know that the word just read was a real word because the phonological representation exists in his or her lexicon. Karamu is an infrequent word, and thus the child would probably give peni ‘pen’ as a synonym. However, in the context of a passage concerning kings, slave traders, and palaces, he or she would recall the less frequent meaning. If given a lexical decision task, the child would identify either form as a word. It is true that he or she might also identify chakura as a word when it is not (chakula is a word: ‘food’). When testing beginning readers, testers and psychologists need to be aware of these issues, just as those working with some English dialects encounter children who think that im or ca is a word because initial /h/ or final /r/ is dropped in some dialects, or those working with Spanish-speaking children in Latin America encounter children who accept lus as a word because s and z are both pronounced /s/ in their dialect. However, given that chakula, him, car, and luz are all regularly spelled words, children speaking these dialects of Kiswahili, English, or Spanish can read these words based on phoneme–grapheme correspondences alone. In a regularly spelled language such as Kiswahili, readers who know all possible phoneme–grapheme correspondences can then use the phonological route to read any word in the language. Once a child has cracked the code, it is theoretically possible for him or her to read every word, familiar and unfamiliar. As there were no standardized reading tests for Kiswahili, it was necessary to develop one. Initially, we hoped to use an adaptation of the Wide Range Achievement Test (WRAT) (Jastak & Jastak, 1965), but following piloting we discovered this was not possible. Oral single-word reading tasks rely on children being able to read aloud only those words that they are actually familiar with in written form. In Kiswahili, as in any regularly spelled language, it is possible for children to appear to read better than they can in reality because they can read out loud words that they do not understand. Hence, normal children will exhibit symptoms of hyperlexia (Ellis, 1993) in an improperly designed test. In research similar to ours carried out in English-speaking developing countries, reading tests used for screening are normally oral single-word reading tasks such as the WRAT (Simeon, Callender, Wong, Grantham-McGregor, & Ramdath, 1994). Other available tests depend on comprehension, such as the Schonell Diagnostic English Tests (Schonell, 1940). Such tests can be useful in some circumstances: for example, if it is explicitly desired to test children’s inference about the meanings of words in continuous text. However, for children at the earlier stages of reading and where it is desired to test decoding skills as well as written word comprehension, these tests are not ideal; in addition, they are difficult to construct well and to use reliably. A good reading test should test whether pupils can both decode and understand the words they are reading. There were additional factors to take into account in constructing these readApplied Psycholinguistics 21:4 530 Alcock et al.: Kiswahili reading tests ing tests. In many countries, the lack of standardized reading tests is a result of little research into reading development in that language, which can be due to a dearth of educational or developmental psychologists working in the country or proficient in the language. This was the case in Tanzania. It was therefore necessary to develop tests that could be used with ease by nonpsychologists as well as by local teachers. In developing countries, large class sizes and lack of equipment are common; thus, an effort was made to develop tests that could be used quickly with large groups of children. In addition, large class sizes and poor facilities often mean that children’s early reading development is so slow as to be almost imperceptible; thus it is desirable to measure some aspects of reading knowledge in preliterate children as well as more advanced skills in older children. Hence, an effort was made to construct a range of reading tests that would be usable across a range of reading skill. The school system in Tanzania has seven years of theoretically compulsory primary education followed by either four or six years of secondary education, which about 5% to 10% of primary school pupils complete. In fact, only around 60% of children of the correct ages enroll in primary school. Reasons for nonenrollment are complex, but may include such things as lack of money for school fees, distance from school, and child labor. Children should start school by age 7; some children enroll in nursery schools before then. Of the schools in our study, two have their own nursery schools, and one is very near a church-run nursery school. Of children in the study, 33% attended nursery school; all were at the three schools with a nursery on site or nearby. Late enrollment is also very common for the same reasons cited for failure to enroll and because parents may be unsure of a child’s age. Late enrollment is more prevalent in rural areas due to higher poverty levels and greater distances to schools. Some children may enroll late purely due to the long distance from home to school, which makes parents reluctant to send a small child to school when this means a long walk each day. The current study was carried out in the first to fifth grades with children aged 8 to 14. All children were within two years of the modal age for their grade.